A Study of Approaches to

Hypertext poses new research challenges for text classiication. Hyperlinks, HTML tags, category labels distributed over linked documents, and meta data extracted from related web sites all provide rich information for classifying hypertext documents. How to appropriately represent that information and automatically learn statistical patterns for solving hypertext classiication problems is an open question. This paper seeks a principled approach to providing the answers. Speciically, we deene ve hypertext regularities which may (or may not) hold in a particular application domain, and whose presence (or absence) may signiicantly innuence the optimal design of a classiier. Using three hypertext datasets and three well-known learning algorithms (Naive Bayes, Nearest Neighbor, and First Order Inductive Learner), we examine these regularities in diierent domains, and compare alternative ways to exploit them. Our experimental results suggest that a naive use of linked pages, such as treating the words in the linked neighborhood of a page as local to that page, can be more harmful than helpful when the linked neighborhood is highly \noisy". This is especially true if the classiier is not suuciently robust in discriminating informative words from noisy ones. It is also evident in our results that extracting meta data (when available) from related web sites can be extremely useful for improving classiication accuracy. Finally, the relative performance of the classiiers being tested provides insights into their strengths and limitations for solving classiication problems involving diverse and often noisy web pages.